Vehicle and method for controlling the same

ABSTRACT

A vehicle may include: a feedback device; a bio-signal sensor configured to measure a bio-signal of a user; and a controller operatively coupled to the feedback device and the bio-signal sensor, the controller including a memory configured to store at least one program instruction and processor configured to execute the at least one program instruction. The controller may be configured to: determine information characterizing a current emotional state of the user based on the bio-signal; calculate, based on a difference value between the current emotional state and a target emotional state, an operation ratio between a first mode for controlling operation of the feedback device to decrease a degree of excitability of the user and a second mode for controlling the operation of the feedback device to increase a degree of positivity of the user; and control the operation of the feedback device for a predetermined time based on the operation ratio.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2019-0001742, filed on Jan. 7, 2019in the Korean Intellectual Property Office, the present disclosure ofwhich is incorporated herein by reference in its entirety.

TECHNICAL FIELD

Embodiments of the present disclosure relate to generally to vehiculartechnologies and, more particularly, to a vehicle with componentssubject to control based on a user's emotion, and a method forcontrolling the vehicle.

BACKGROUND

In recent years, technologies for detecting an emotional state of avehicle user have been actively studied. Also, similar technologies forinducing a positive emotion in a vehicle user have been conducted.

However, conventional techniques are limited merely to determining thepositivity or negativity of the user's emotional state. Thus, suchtechniques are only capable of providing feedback that regulates theoutput of vehicular components based on whether the determined emotionalstate of the user is positive or negative.

SUMMARY

Therefore, it is an aspect of the present disclosure to provide avehicle capable of determining a mode in which a user's emotional statein the vehicle is determined as a degree of positivity and a degree ofexcitability. Furthermore, it is an aspect of the present disclosure toprovide a vehicle capable of determining an operation ratio between amode for controlling a feedback device that decreases the degree ofexcitability and a mode for controlling the feedback device thatincreases the degree of positivity according to the determined degree ofpositivity and degree of excitability. Furthermore, it is an aspect ofthe present disclosure to provide a method for controlling the vehicle.

Additional aspects of the present disclosure will be set forth in partin the description which follows and, in part, will be apparent from thedescription, or may be learned by practice of the present disclosure.

In accordance with embodiments of the present disclosure, a vehicle mayinclude: a feedback device; a bio-signal sensor configured to measure abio-signal of a user; and a controller operatively coupled to thefeedback device and the bio-signal sensor, the controller including amemory configured to store at least one program instruction andprocessor configured to execute the at least one program instruction.The controller may be configured to: determine informationcharacterizing a current emotional state of the user based on thebio-signal; calculate, based on a difference value between the currentemotional state and a target emotional state, an operation ratio betweena first mode for controlling operation of the feedback device todecrease a degree of excitability of the user and a second mode forcontrolling the operation of the feedback device to increase a degree ofpositivity of the user; and control the operation of the feedback devicefor a predetermined time based on the operation ratio.

The controller may control the feedback device, based on the operationratio, such that the feedback device performs the first mode for a firsttime of the predetermined time and performs the second mode for a secondtime of the predetermined time, wherein the second time equals adifference between the predetermined time and the first time.

The controller may control the feedback device such that the first modeand the second mode are repeatedly alternately performed based on apredetermined number of mode switching times and a predetermined holdingtime for each mode.

The controller may perform an operation through a neural network basedon the current emotional state and the target emotional state; determinethe number of mode switching times and the holding time for each modecorresponding to the current emotional state and the target emotionalstate based on information characterizing the operation performedthrough the neural network; and control the operation of the feedbackdevice such that the first mode and the second mode are repeatedlyalternately performed based on the determined number of mode switchingtimes and the determined holding time for each mode.

The controller may determine the degree of excitability of the user andthe degree of positivity of the user based on the current emotionalstate, to compare the degree of excitability of the user with the degreeof positivity of the user, and to perform either the first mode or thesecond mode based on the comparison of the degree of excitability of theuser with the degree of positivity of the user.

The controller may calculate a first difference value between the degreeof excitability for the current emotional state and the degree ofexcitability for the target emotional state calculate, to calculate asecond difference value between the degree of positivity for the currentemotional state and the degree of positivity for the target emotionalstate calculate, and to calculate the operation ratio between the firstmode and the second mode as equivalent to a ratio between the firstdifference value and the second difference value.

The feedback device may be disposed in the vehicle and include at leastone of a speaker, a display, an air conditioner, and a vibration deviceinstalled in a seat.

The controller may extract an emotion factor that affects the currentemotional state; in the first mode, control the operation of thefeedback device causing an emotion factor to decrease according to thedegree of excitability of the extracted emotion factor; and in thesecond mode, control the operation of the feedback device causing anemotion factor to increase according to the degree of positivity of theextracted emotion factor.

Furthermore, in accordance with embodiments of the present disclosure, amethod for controlling a vehicle, the vehicle including a feedbackdevice, a bio-signal sensor configured to measure a bio-signal of auser, and a controller operatively coupled to the feedback device andthe bio-signal sensor, may include: determining, by the controller,information characterizing a current emotional state of the user basedon the bio-signal; calculating, by the controller, based on a differencevalue between the current emotional state and a target emotional state,an operation ratio between a first mode for controlling the operation ofthe feedback device to decrease a degree of excitability of the user anda second mode for controlling the operation of the feedback device toincrease a degree of positivity of the user; and controlling, by thecontroller, the operation of the feedback device for a predeterminedtime based on the operation ratio.

The controlling of the feedback device may include controlling theoperation of the feedback device, based on the operation ratio, suchthat the feedback device performs the first mode for a first time of thepredetermined time and performs the second mode for a second time of thepredetermined time, wherein the second time equals a difference betweenthe predetermined time and the first time.

The controlling of the feedback device may include controlling theoperation of the feedback device such that the first mode and the secondmode are repeatedly alternately performed based on a predeterminednumber of mode switching times and a predetermined holding time for eachmode.

The controlling of the feedback device may include performing anoperation through a neural network based on the current emotional stateand the target emotional state; determining the number of mode switchingtimes and the holding time for each mode corresponding to the currentemotional state and the target emotional state based on informationcharacterizing the operation performed through the neural network; andcontrolling the operation of the feedback device such that the firstmode and the second mode are repeatedly alternately performed based onthe determined number of mode switching times and the determined holdingtime for each mode.

The controlling of the feedback device may include determine the degreeof excitability of the user and the degree of positivity of the userbased on the current emotional state; comparing the degree ofexcitability of the user with the degree of positivity of the user; andperforming either the first mode or the second mode based on thecomparison of the degree of excitability of the user with the degree ofpositivity of the user.

The calculating of the operation ratio may include calculating a firstdifference value between the degree of excitability for the currentemotional state and the degree of excitability for the target emotionalstate calculate; calculating a second difference value between thedegree of positivity for the current emotional state and the degree ofpositivity for the target emotional state calculate; and calculating theoperation ratio between the first mode and the second mode as equivalentto a ratio between the first difference value and the second differencevalue.

The feedback device may be disposed in the vehicle and include at leastone of a speaker, a display, an air conditioner, and a vibration deviceinstalled in a seat.

The controlling of the feedback device may include extracting an emotionfactor that affects the current emotional state; in the first mode,controlling the operation of the feedback device causing an emotionfactor to decrease according to the degree of excitability of theextracted emotion factor; and in the second mode, controlling theoperation of the feedback device causing an emotion factor to increaseaccording to the degree of positivity of the extracted emotion factor.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects of the present disclosure will becomeapparent and more readily appreciated from the following description ofthe embodiments, taken in conjunction with the accompanying drawings ofwhich:

FIG. 1 is a control block diagram of a vehicle according to embodimentsof the present disclosure;

FIG. 2 is a view illustrating correlation information betweenbio-signals and emotion factors according to embodiments of the presentdisclosure;

FIG. 3 is a view illustrating correlation information between facialexpressions and emotion factors according to embodiments of the presentdisclosure;

FIG. 4 is a view illustrating an emotion model according to embodimentsof the present disclosure;

FIG. 5 is a view illustrating a difference value between a currentemotional state and a target emotional state according to embodiments ofthe present disclosure;

FIG. 6 is a view illustrating mode switching of a feedback deviceaccording to embodiments of the present disclosure;

FIG. 7 is a view illustrating control of the feedback device accordingto each mode in the vehicle according to embodiments of the presentdisclosure;

FIG. 8 is a flowchart illustrating a method of controlling the feedbackdevice in a method of controlling the vehicle according to embodimentsof the present disclosure; and

FIG. 9 is another flowchart illustrating a method of controlling thefeedback device in a method of controlling the vehicle according toembodiments of the present disclosure.

It should be understood that the above-referenced drawings are notnecessarily to scale, presenting a somewhat simplified representation ofvarious preferred features illustrative of the basic principles of thepresent disclosure. The specific design features of the presentdisclosure, including, for example, specific dimensions, orientations,locations, and shapes, will be determined in part by the particularintended application and use environment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the accompanying drawings. As those skilled inthe art would realize, the described embodiments may be modified invarious different ways, all without departing from the spirit or scopeof the present disclosure. Like numbers refer to like elementsthroughout this specification. This specification does not describe allcomponents of the embodiments, and general information in the technicalfield to which the present disclosure belongs or overlapping informationbetween the embodiments will not be described.

It will be understood that when a component is referred to as being“connected” to another component, it can be directly or indirectlyconnected to the other component. When a component is indirectlyconnected to another component, it may be connected to the othercomponent through a wireless communication network.

Also, it will be understood that the terms “includes,” “comprises,”“including,” and/or “comprising,” when used in this specification,specify the presence of a stated component, but do not preclude thepresence or addition of one or more other components.

Also, it is to be understood that the singular forms “a,” “an,” and“the” include plural referents unless the context clearly dictatesotherwise.

As used herein, the terms “portion,” “unit,” “block,” “member,” or“module” refer to a unit that can perform at least one function oroperation. For example, these terms may refer to at least one processwhich is performed by at least one piece of hardware such as afield-programmable gate array (FPGA) and an application specificintegrated circuit (ASIC), and at least one piece of software stored ina memory, or a processor.

Reference numerals used in operations are provided for convenience ofdescription, without describing the order of the operations, and theoperations can be executed in a different order from the stated orderunless a specific order is definitely specified in the context.

It is understood that the term “vehicle” or “vehicular” or other similarterm as used herein is inclusive of motor vehicles in general such aspassenger automobiles including sports utility vehicles (SUV), buses,trucks, various commercial vehicles, watercraft including a variety ofboats and ships, aircraft, and the like, and includes hybrid vehicles,electric vehicles, plug-in hybrid electric vehicles, hydrogen-poweredvehicles and other alternative fuel vehicles (e.g., fuels derived fromresources other than petroleum). As referred to herein, a hybrid vehicleis a vehicle that has two or more sources of power, for example bothgasoline-powered and electric-powered vehicles.

Additionally, it is understood that one or more of the below methods, oraspects thereof, may be executed by at least one controller. The term“controller” may refer to a hardware device that includes a memory and aprocessor. The memory is configured to store program instructions, andthe processor is specifically programmed to execute the programinstructions to perform one or more processes which are describedfurther below. The controller may control operation of units, modules,parts, devices, or the like, as described herein. Moreover, it isunderstood that the below methods may be executed by an apparatuscomprising the controller in conjunction with one or more othercomponents, as would be appreciated by a person of ordinary skill in theart.

Furthermore, the controller of the present disclosure may be embodied asnon-transitory computer readable media containing executable programinstructions executed by a processor. Examples of the computer readablemediums include, but are not limited to, ROM, RAM, compact disc(CD)-ROMs, magnetic tapes, floppy disks, flash drives, smart cards andoptical data storage devices. The computer readable recording medium canalso be distributed throughout a computer network so that the programinstructions are stored and executed in a distributed fashion, e.g., bya telematics server or a Controller Area Network (CAN).

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the accompanying drawings.

FIG. 1 is a control block diagram of a vehicle according to embodimentsof the present disclosure.

As shown in FIG. 1 , a vehicle 10 according to embodiments of thepresent disclosure may include a camera 110 for capturing a userentering the vehicle 10 and obtaining image data of the user, abio-signal sensor 120 for measuring a bio-signal of the user, aninputter 130 for receiving an input from the user, a controller 140 fordetermining the user's current emotional state based on at least one ofthe image data of the user or the bio-signal and determining anoperation ratio between a first mode for controlling a feedback device150 so as to decrease a degree of excitability of the user and a secondmode for controlling the feedback device 150 so as to increase thedegree of positivity of the user based on a difference value between thedetermined current emotional state and a target emotional state, thefeedback device 150 provided in the vehicle 10 for outputting feedbackthat causes the user's emotion to be a target emotion under the controlof the controller 140, and a storage 160 for storing various informationrequired for controls of the vehicle 10.

The camera 110 may capture the user in the vehicle 10 to obtain theimage data of the user. The image data of the user may includeinformation characterizing the user's facial expression, i.e., a motionof the facial body composition.

The camera 110 may include a plurality of lenses and an image sensor.The image sensor may include a plurality of photodiodes for convertinglight into electrical signals, and the plurality of photodiodes may bearranged in a two-dimensional matrix.

In addition, the camera 110 may be an infrared camera for capturing theuser during night driving. Also in this case, the image data of the usermay include information characterizing the user's facial expression.

The camera 110 may be installed on a dashboard, a windshield or a seatof the vehicle 10. However, there is no limitation on the installationposition and number of the cameras 110. The image data of the user,obtained by the camera 110, may be used by the controller 140 toidentify the user's facial expression and the user's position.Therefore, the camera 110 may be installed in front of the user.

At this time, the user may include both a driver and a passenger of thevehicle 10. The camera 110 may obtain the image data of each of theusers in the vehicle 10.

The bio-signal sensor 120 according to embodiments of the presentdisclosure may measure a bio-signal of each of the users in the vehicle10. The bio-signal of the user may be transmitted to the controller 140and stored in the storage 160.

The bio-signal sensor 120 may be installed at various positions in thevehicle 10. For example, the bio-signal sensor 120 may be provided onthe seat, a seat belt, a steering wheel, and a handle provided on adoor.

In addition, the bio-signal sensor 120 may be provided as a kind ofwearable device that the user who entered the vehicle 10 can wear.

The bio-signal sensor 120 may include at least one of a galvanic skinresponse (GSR) sensor for measuring skin electrical conductivitydepending on a sweat rate of the user, a skin temperature sensor formeasuring a skin temperature of the user, a heart rate (HR) sensor formeasuring a heart rate of the user, an electroencephalogram (EEG) sensorfor measuring brainwaves of the user, a voice recognition sensor formeasuring a voice signal of the user, a blood pressure measurementsensor for measuring a blood pressure of the user, or an eye tracker fortracking the positions of pupils. However, sensors that may be includedin the bio-signal sensor 120 are not limited to the above-mentionedsensors, and the bio-signal sensor 120 may include another sensorcapable of measuring a person's bio-signal.

The inputter 130 according to embodiments of the present disclosure mayreceive an input from the user. Particularly, the inputter 130 mayreceive an input for the target emotional state, an input for a drivemode (e.g., the first mode and the second mode) of the feedback device150, and an input for a mode switching pattern (e.g., number of modeswitching times, mode holding times), etc. from the user.

For this, the inputter 130 may be provided in a center fascia (notshown) installed at the center of a dashboard and may be implementedwith mechanical buttons, knobs, touch pad, touch screen, stick-typemanipulation device, trackball, or the like. At this time, the inputter130 disposed on the touch screen may be provided on a display 152provided inside the vehicle 10. However, the position and implementationmethod of the inputter 130 are not limited to the above-describedexample, and may be included without limitation as long as the positionand the implementation method in which the user's input can be received.

The controller 140 according to embodiments of the present disclosuremay determine the user's current emotional state based on at least oneof the image data of the user or the bio-signal, and control thefeedback device 150 based on the determined current emotional state.

Particularly, the controller 140 may determine a facial expression ofthe user based on the image data of the user, and obtain informationcharacterizing an emotional state corresponding to the facialexpression. A configuration for obtaining information characterizing theuser's emotional state based on image data will be described in detaillater.

In addition, the controller 140 may obtain information characterizingthe user's emotional state corresponding to the bio-signal of the userbased on the bio-signal of a passenger 200. A configuration forobtaining information characterizing the user's emotional state based onthe bio-signal will be described in detail later.

The controller 140 may determine the user's current emotional statebased on at least one of the image data of the user or the bio-signal,and control the feedback device 150 based on the determined user'scurrent emotional state.

Particularly, the controller 140 may compare the degree of excitabilityof the user and the degree of positivity of the user indicated by thecurrent emotional state, and select either the first mode forcontrolling the feedback device 150 in a direction of decreasing thedegree of excitability of the user or the second mode for controllingthe feedback device 150 in a direction of increasing the degree ofpositivity of the user based on the comparison result. That is, thecontroller 140 may determine the operation ratio between the first modeand the second mode to 100 to 0 or 0 to 100 for immediate feedback onthe current emotional state according to the embodiment. Accordingly,the controller 140 may control the feedback device 150 to operate ineither the first mode or the second mode.

That is, the controller 140 may determine the user's current emotionalstate by quantifying the degree of excitability and degree ofpositivity. A detailed explanation will be described later.

In this case, when a numerical value of the degree of excitabilityindicated by the current emotional state is higher than the numericalvalue of the degree of negativity indicated by the current emotionalstate, the controller 140 may control the feedback device 150 to operatein the first mode. When the numerical value of the degree ofexcitability indicated by the current emotional state is lower than thenumerical value of the degree of negativity indicated by the currentemotional state, the controller 140 may control the feedback device 150to operate in the second mode.

At this time, the numerical value of the degree of negativity maycorrespond to the numerical value of the degree of positivity having aminus (−) value. In the comparison between the numerical value of thedegree of excitability and the numerical value of the degree ofnegativity, it is premised that the comparison is made based on anabsolute value of each numerical value.

In addition, the controller 140 may determine the operation ratiobetween the first mode and the second mode based on the difference valuebetween the user's current emotional state and the target emotionalstate, and control the feedback device 150 based on the determinedoperation ratio for a predetermined time.

The controller 140 may determine the difference value between the user'scurrent emotional state and the target emotional state based on at leastone of the image data of the user and the bio-signal. The targetemotional state may be set when the vehicle 10 is designed, or may beset based on the user's input through the inputter 130.

The difference value between the user's current emotional state and thetarget emotional state may include a difference value for the degree ofexcitability and a difference value for the degree of positivity. Thatis, the controller 140 may compare the degree of excitability and thedegree of positivity of the target emotional state on the basis of thedetermined degree of excitability and the determined degree ofpositivity, respectively, and determine the difference value for thedegree of excitability and the difference value for the degree ofpositivity.

At this time, the controller 140 may determine the operation ratiobetween the first mode and the second mode so as to correspond to theratio between the difference value for the degree of excitability andthe difference value for the degree of positivity between the currentemotional state and the target emotional state.

Thereafter, the controller 140 may control the feedback device 150 toperform the first mode for a first time of the predetermined time. Thecontroller 140 may control the feedback device 150 to perform the secondmode for a second time of the predetermined time, the second timeequaling a difference between the predetermined time and the first time.

The predetermined time may be set when the vehicle 10 is designed, ormay be set based on the user's input through the inputter 130. Inaddition, the predetermined time may correspond to an estimated drivingtime to a destination calculated by the controller 140 according to theuser's destination input through the inputter 130.

The controller 140 may control the feedback device 150 such that thefirst mode and the second mode are alternately performed, repeatedly,based on a predetermined number of mode switching times and apredetermined holding time for each mode.

The predetermined number of mode switching times and the predeterminedholding time for each mode may be set when the vehicle 10 is designed,or may be set based on the user's input through the inputter 130.

That is, the controller 140 may control the feedback device 150 suchthat the first mode and the second mode are repeatedly alternatelyperformed. At this time, the total time that the feedback device 150operates in the first mode may correspond to the first time of thepredetermined time, and the total time that the feedback device 150operates in the second mode may correspond to a second time of thepredetermined time, the second time equaling a difference between thepredetermined time and the first time.

In addition, the controller 140 may perform an operation through aneural network on the current emotional state and the target emotionalstate, determine the number of mode switching times and the holding timefor each mode corresponding to the current emotional state and thetarget emotional state based on information characterizing the operationperformed through the neural network, and control the feedback device150 such that the first mode and the second mode are repeatedlyalternately performed based on the determined number of mode switchingtimes and the determined holding time for each mode.

Particularly, the controller 140 may perform the operation through theneural network on the determined current emotional state and the targetemotional state, determine the number of mode switching times, anddetermine an operation pattern of the feedback device 150 based on theinformation characterizing the operation performed through the neuralnetwork.

At this time, the operation pattern of the feedback device 150 maycorrespond to the mode in which the feedback device 150 operates byswitching the mode for the predetermined time based on the number oftimes an operation mode is switched for the predetermined time and theholding time for each mode.

In this case, the information characterizing the operation performedthrough the neural network may include information characterizing thenumber of mode switching times indicating the number of times the modeis switched during the predetermined time and information characterizingthe holding time for each mode indicating a time at which the mode foreach mode to be switched is held.

The above-mentioned neural network refers to machine learning that formsa neural structure capable of performing deep learning, so that a weightand bias corresponding to the configuration of the neural networkcontinuously changes, thereby improving the reliability of learning.

Particularly, the vehicle 10 may improve the inference result of theneural network by continuously updating the weight, bias and activationfunction included in the neural network based on the current emotionalstate, the target emotional state, and information characterizing anarrival time of the user's emotion according to the pattern of thefeedback device 150 to the target emotion. That is, the vehicle 10 maystore the determined pattern and the information characterizing thearrival time of the user's emotion to the target emotion whenever thevehicle 10 drives, and continuously update the stored neural networkbased on information characterizing the stored determined pattern andthe information characterizing the arrival time.

At this time, the neural network may be stored in the storage 160 in theform of a computer program. Hereinafter, the operation performed by theneural network in the coding form of the computer program will bedescribed. However, the present disclosure is not limited to thecomputer program in which the neural network is stored.

Meanwhile, the neural network may include a Convolution Neural Network(CNN) that generates a feature map output by convoluting the currentemotional state and the target emotional state, and inputs the featuremap to the neural network. However, the neural network may be performedwith other deep-running algorithms including Recurrent Neural Networks(RNN).

In this way, the vehicle 10 may determine the operation pattern of thefeedback device 150 that is most suitable based on the current emotionalstate and the target emotional state through the neural network that iscontinuously updated according to the use. That is, the controller 140may determine the number of mode switching times and the holding timefor each mode in which the user's emotion can quickly reach the targetemotion, based on the current emotional state and the target emotionalstate.

However, the controller 140 may control the feedback device 150 tooperate in either the first mode or the second mode based on the user'sinput through the inputter 130.

The controller 140 may control the feedback device 150 based on theemotion of any one of the users selected through the inputter 130 fromthe plurality of users who entered the vehicle 10.

In addition, the controller 140 may control the feedback device 150based on the user's emotion corresponding to the current emotional statehaving the highest difference value from the target emotional state,based on the current emotional state of each of the plurality of usersin the vehicle 10.

Hereinafter, the control of the feedback device 150 of the controller140 according to the modes will be described in detail.

The controller 140 may extract an emotion factor that affects the user'scurrent emotional state and control the feedback device 150 in adirection of decreasing the emotion factor corresponding to the degreeof excitability of the extracted emotion factor in the first mode, andcontrol the feedback device 150 in a direction of increasing the emotionfactor corresponding to the degree of positivity of the extractedemotion factors in the second mode.

The feedback device 150 may include at least one of a speaker 151, thedisplay 152, an air conditioner 154 provided in the vehicle 10, and avibration device 153 provided in the seat of the vehicle 10.

The speaker 151 may output sound to the inside of the vehicle 10 havinga frequency corresponding to a target breathing cycle or a targetheartbeat cycle in the first mode under the control of the controller140, or output sound to the inside of the vehicle 10 that increases theemotion factor corresponding to the degree of positivity in the secondmode under the control of the controller 140.

That is, the controller 140 may set a breathing cycle and a heartbeatcycle when the target emotional state is the target breathing cycle andthe target heartbeat cycle, respectively, and control the speaker 151 tooutput sound corresponding to the target breathing cycle or the targetheartbeat cycle to the inside of the vehicle 10 so that the degree ofexcitability of the user is decreased in the first mode. Accordingly,the vehicle 10 can cause the user's breathing cycle to be induced to thetarget breathing cycle or the user's heartbeat cycle to be induced tothe target heartbeat cycle.

At this time, the sound that increases the emotion factor correspondingto the degree of positivity may be set when the vehicle 10 is designed,and may correspond to sound that has at least one of a size, a genre, anequalizer, a tone color, and a sound wave region of the sound in whichthe user can feel a positive emotion. For example, the sound thatincreases the emotion factor corresponding to the degree of positivitymay include hip-hop music, classical music, and pop music in which theuser can feel the positive emotion. However, the sound that increasesthe emotion factor corresponding to the degree of positivity may be setbased on the user's input through the inputter 130. That is, the usermay set the sound that caused the positive emotion to be the sound thatincreases the emotion factor corresponding to the degree of positivity,and the sound that is set can be stored in the storage 160.

For this, the speaker 151 may be provided inside the vehicle 10, and maybe provided without limitation as long as it is in a position where theuser can listen to the sound outputted.

The display 152 according to embodiments of the present disclosure mayoutput an image having a frequency corresponding to the target breathingcycle or the target heartbeat cycle to the inside of the vehicle 10 inthe first mode, and output the image having the frequency correspondingto the sound for increasing the emotion factor corresponding to thedegree of positivity to the inside of the vehicle 10 in the second mode,under the control of the controller 140.

More particularly, the controller 140 may control the display 152 toincrease or decrease the intensity of light output according to thefrequency corresponding to the target breathing cycle or the targetheartbeat cycle in the first mode, and control the display 152 toincrease or decrease the intensity of the light output according to thefrequency corresponding to the sound that increases the emotion factorcorresponding to the degree of positivity in the second mode.

The display 152 may also output an image of a color that decreases theemotion factor corresponding to the degree of excitability in the firstmode based on the control of the controller 140, and output the image ofthe color that increases the emotion factor corresponding to thepositivity of the image in the second mode, under the control of thecontroller 140.

At this time, the color for decreasing the emotion factor correspondingto the degree of excitability and the color for increasing the emotionfactor corresponding to the degree of positivity may be set when thevehicle 10 is designed, may be set by the controller 140 based oncorrelation information between the color and the emotion factorobtained from an external server, and may be set by the user through theinputter 130.

For this, the display 152 may be provided within the vehicle 10 and mayinclude a panel. For example, the display 152 may be provided in acluster, provided across the cluster and the center fascia, and providedin a ceiling or the door inside the vehicle 10.

The panel may be at least one of a cathode ray tube (CRT) panel, aliquid crystal display (LCD) panel, a light emitting diode (LED) panel,an organic light emitting diode (OLED) panel, a plasma display panel(PDP), and a field emission display (FED) panel.

The position and number of the displays 152 may be included withoutlimitation as long as the position and number of the pieces ofinformation can be visually transmitted to the user of the vehicle 10.

The vibration device 153 according to embodiments of the presentdisclosure may output a vibration having a frequency corresponding tothe target breathing cycle or the target heartbeat cycle in the firstmode, or output the vibration having the frequency corresponding to thesound for increasing the emotion factor corresponding to the degree ofpositivity in the second mode, under the control of the controller 140.

At this time, the outputted vibration causes the seat provided in thevehicle 10 to vibrate, and may be transmitted to the user located on theseat through the seat.

For this, the vibration device 153 may be provided in the seat providedin the vehicle 10, and may correspond to a motor driven under thecontrol of the controller 140.

At this time, the vibration device 153 may rotate to correspond to thefrequency transmitted from the controller 140 and transmit the vibrationcorresponding to the frequency to the user on the seat.

Further, according to the embodiment, the vibration device 153 may alsoinclude a diaphragm for amplifying the vibration.

The air conditioner 154 may output at least one of a perfume and windthat decreases the emotion factor corresponding to the degree ofexcitability in the first mode, and output at least one of the perfumeand wind that increases the emotion factor corresponding to the degreeof positivity in the second mode, under the control of the controller140.

At this time, the perfume and wind that decrease the emotion factorcorresponding to the degree of excitability may be set when the vehicle10 is designed, may be set by the controller 140 based on thecorrelation information between the perfume and the emotion factorobtained from the external server, and may be set by the user throughthe inputter 130. For example, the perfume and wind that decrease theemotion factor corresponding to the degree of excitability maycorrespond to jasmine perfume and a breeze, respectively. However, thepresent disclosure is not limited thereto, and any perfume that candecrease the emotion factor corresponding to the degree of excitabilitymay be included without limitation. Also, the wind that decreases theemotion factor corresponding to the degree of excitability maycorrespond to wind having at least one of wind direction, intensity, andtemperature that can lower the degree of excitability.

In addition, the perfume and wind that decrease the emotion factorcorresponding to the degree of positivity may be set when the vehicle 10is designed, may be set by the controller 140 based on the correlationinformation between the perfume and the emotion factor obtained from theexternal server, and may be set by the user through the inputter 130.For example, the perfume and wind that increase the emotion factorcorresponding to the degree of positivity may correspond to lemonperfume and a cool breeze, respectively. However, the present disclosureis not limited thereto, and any perfume that can increase the emotionfactor corresponding to the degree of positivity may be included withoutlimitation. Also, the wind that increases the emotion factorcorresponding to the degree of positivity may correspond to wind havingat least one of wind direction, intensity, and temperature that canraise the degree of positivity.

For this, the air conditioner 154 may be provided in the vehicle 10 toblow wind (warm air or cool air) to the indoor space of the vehicle 10under the control of the controller 140.

More particularly, the air conditioner 154 may include a compressor, acondenser, an expansion valve, and a heat exchanger, and the compressor,the condenser, the expansion valve, and the heat exchanger may beconnected to each other through at least one refrigerant passage. Arefrigerant may flow through the compressor, the condenser, theexpansion valve, and the heat exchanger along the refrigerant passage,and the air conditioner 154 may obtain cold air or warm air depending ona change in the state of the flowing refrigerant. The cold air or warmair may be provided to the indoor space of the vehicle 10 through a fan.

The air conditioner 154 may include a motor for driving the fan thatgenerates wind blowing into the indoor space of the vehicle 10, and alsoinclude a motor for adjusting a wing member (not shown) provided in airvents 141 to adjust a direction of the blowing wind.

In addition, the air conditioner 154 may include an aroma actuator thatinjects aroma substances in wind blowing into the indoor space, and thearoma actuator may include a plurality of storage tanks for storingvarious kinds of aroma substances, an injection port for injecting aromasubstances, and a motor for injecting aroma substances through theinjection port.

In this way, the controller 140 may control at least one of the speaker151, the display 152, the vibration device 153, and the air conditioner154 corresponding to the feedback device 150 to induce the user'spositive emotion and decrease the degree of excitability of the user.

The controller 140 may include at least one memory storing a program forperforming the above-described operations and operations which will bedescribed below, and at least one processor for executing the storedprogram. When there are a plurality of memories and processors, they maybe integrated into one chip or provided at physically separatedpositions.

The storage 160 according to embodiments of the present disclosure maystore various information required for controls of the vehicle 10.

For example, the storage 160 may store the image data obtained by thecamera 110, a measured value obtained by the bio-signal sensor 120,correlation information between the user's bio-signal and an emotionfactor, correlation information between the user's facial expression andan emotion factor, the user's emotion information, an emotion model, andthe like. Data stored in the storage 160 may be transmitted to thecontroller 140.

The storage 160 may be implemented as at least one of a non-volatilememory device (for example, a cache, Read Only Memory (ROM),Programmable ROM (PROM), Erasable Programmable ROM (EPROM), ElectricallyErasable Programmable ROM (EEPROM), and flash memory), a volatile memorydevice (for example, Random Access Memory (RAM)), or storage medium,such as Hard Disk Drive (HDD) and Compact Disc Read Only Memory(CD-ROM), although not limited to these. The storage 160 may be a memoryimplemented as a separate chip, or the storage 160 and the processor maybe integrated into a single chip.

Hereinafter, the vehicle 10 for obtaining information characterizing theuser's emotional state based on the image data obtained by the vehicle10 through the camera 110 and the bio-signal obtained through thebio-signal sensor 120 will be described in detail.

FIG. 2 is a view illustrating correlation information betweenbio-signals and emotion factors according to embodiments of the presentdisclosure, FIG. 3 is a view illustrating correlation informationbetween facial expressions and emotion factors according to embodimentsof the present disclosure, and FIG. 4 is a view illustrating an emotionmodel according to embodiments of the present disclosure.

Referring first to FIG. 2 , correlation information 200 between thebio-signal and the emotion factor may include correlation informationbetween the GSR and the EEG and emotion factors.

As illustrated in FIG. 2 , a GSR signal has correlation values of 0.875and 0.775 with emotion factors of Disgust and Anger, respectively, whichindicates that the GSR signal has a high relevance with the emotionfactors of Disgust and Anger. Therefore, a bio-signal of the usercollected by a GSR sensor may be the basis on which an emotion of theuser is determined as an anger emotion or a disgust emotion.

An emotion factor of Joy has a relatively low correlation value of 0.353with a GSR signal, which indicates that the emotion factor of Joy isless relevant to the GSR signal.

An EEG signal has correlation values of 0.864 and 0.878 with emotionfactors of Anger and Fear, respectively, which indicates that the EEGsignal has a relatively higher relevance to the emotion factors of Angerand Fear than the other emotion factors. Therefore, a bio-signalcollected by a EEG sensor may be the basis on which an emotion of theuser is determined as an anger emotion or a fear emotion.

In this way, the controller 140 may obtain emotion information of eachof the users by using the correlation information 200 betweenbio-signals and emotion factors. However, because the informationillustrated in FIG. 2 is of experimental results, it may vary dependingon experimental environments.

Although correlation information between the GSR and the EEG and emotionfactors is illustrated in FIG. 2 , the correlation information 200between the bio-signals and the emotion factors may include correlationinformation between different bio-signals and emotion factors dependingon the kinds of bio-signals measured by sensors provided in the vehicle10.

Referring next to FIG. 3 , the controller 140 according to embodimentsof the present disclosure may recognize a facial expression of each ofthe users appearing on an image of the user captured by the camera 110,and obtain emotion information of the user by applying a facial actioncoding system (FACS) to the facial expression of the user.

More particularly, the controller 140 may extract a plurality of featurepoints from a face of the user, and extract a plurality of facialelements by using the extracted feature points. The plurality of facialelements may include eyebrows, eyes, nose, mouth, and the like. Thecontroller 140 may combine patterns of the plurality of facial elements,and compare the combined pattern with correlation information 300between facial expressions and emotion factors stored in the storage160. The correlation information 300 between facial expressions andemotion factors may correspond to information representing relationshipsbetween facial expressions and emotion factors.

The controller 140 may determine a facial expression corresponding tothe same pattern as or the most similar pattern to the combined patternof the user in the correlation information 300 between facialexpressions and emotion factors, and determine the searched facialexpression as a facial expression of the user.

In addition, the controller 140 may obtain emotion informationrepresenting an emotion of the user by considering a correlation valuefor the determined facial expression of the user in the correlationinformation 300 between facial expressions and emotion factors.

For example, when the determined facial expression of the usercorresponds to a facial expression 2 in the correlation information 300between facial expressions and emotion factors, the controller 140 mayobtain emotion information representing that an emotion of the user is ajoy emotion having a highest correlation value for the facial expression2.

In FIG. 3 , the correlation information 300 between facial expressionsand emotion factors includes a facial expression 1, the facialexpression 2, and a facial expression 3, however, the correlationinformation 300 may further include another facial expression that mayrepresent a passenger's emotion.

In this way, the controller 140 may analyze an image of the inside ofthe vehicle 10, photographed by the camera 110, to determine a facialexpression of each of the users in the vehicle 10 and to obtain emotioninformation of the user based on the determined facial expression.

Referring next to FIG. 4 , an emotion model 400 may be a graphillustrating emotions of the user classified according to image data ofthe user and bio-signals of the user.

The emotion model 400 may classify the emotions of the user on the basisof predetermined emotion axes. The emotion axes may be determined basedon emotions measured from images of the user or from bio-signals of theuser. For example, emotional axis 1 may be degrees of positivity ornegativity, which are measurable by voices or facial expressions of theuser, and emotional axis 2 may be degrees of excitability or activity,which are measurable by the GSR or the EEG.

When an emotion of the user has a high degree of positivity and a highdegree of excitability, the emotion may be classified to emotion 1 oremotion 2. Conversely, when an emotion of the user has minus (−)positivity, i.e., a high degree of negativity and a high degree ofexcitability, the emotion may be classified to emotion 3 or emotion 4.

The emotion model may be a Russell's emotion model. The Russell'semotional model may be expressed by a two-dimensional graph based on thex-axis and the y-axis, and may classify emotions to eight areas of joy(0 degrees), excitement (45 degrees), arousal (90 degrees), pain (135degrees), unpleasantness (180 degrees), depression (225 degrees),sleepiness (270 degrees), and relaxation (315 degrees). In addition, theeight areas may comprise a total of 28 emotions that are classified intosimilar emotions belonging to the eight areas.

In this way, the controller 140 may obtain emotion information of eachof the users by using facial expressions and bio-signals of the user,the correlation information 200 between bio-signals and emotion factors,the correlation information 300 between facial expressions and emotionfactors, and the emotion model 400.

At this time, the obtained emotion information may be defined asnumerical values for the degree of positivity and degree of excitabilityaccording to the degrees of emotions. Particularly, each degree ofpositivity and degree of excitability may be expressed numerically as avalue between −100 and 100 depending on the degrees. However, thenumerical values according to the degrees are merely an example, and maybe included without limitation as long as it corresponds to numericalvalues that can represent the degrees.

Hereinafter, the manner in which the vehicle 10 according to embodimentsof the present disclosure controls the feedback device 150 will bedescribed in detail.

FIG. 5 is a view illustrating a difference value between a currentemotional state and a target emotional state according to embodiments ofthe present disclosure, FIG. 6 is a view illustrating mode switching ofa feedback device according to embodiments of the present disclosure,and FIG. 7 is a view illustrating control of the feedback deviceaccording to each mode in the vehicle according to embodiments of thepresent disclosure.

Referring first to FIG. 5 , the controller 140 may determine the user'scurrent emotional state based on at least one of image data of the userand bio-signals.

For example, the determined user's current emotional state may beclassified as emotion 8, which has a high degree of excitability and ahigh degree of negativity. That is, the information characterizing thecurrent emotional state may indicate plus (+) excitability and minus (−)positivity, and each degree of positivity and degree of excitability maybe expressed numerically according to the degrees.

The controller 140 may control the feedback device 150 based on thedetermined current emotional state.

Particularly, the controller 140 may compare the degree of excitabilityof the user and the degree of positivity of the user indicated by thecurrent emotional state, and select either the first mode forcontrolling the feedback device 150 in a direction of decreasing thedegree of excitability of the user or the second mode for controllingthe feedback device 150 in a direction of increasing the degree ofpositivity of the user based on the comparison result. That is, thecontroller 140 may determine the operation ratio between the first modeand the second mode to 100 to 0 or 0 to 100 for immediate feedback onthe current emotional state according to the embodiment. Accordingly,the controller 140 may control the feedback device 150 to operate ineither the first mode or the second mode.

In this case, when the numerical value of the degree of excitabilityindicated by the current emotional state is higher than the numericalvalue of the degree of negativity indicated by the current emotionalstate, the controller 140 may control the feedback device 150 to operatein the first mode. When the numerical value of the degree ofexcitability indicated by the current emotional state is lower than thenumerical value of the degree of negativity indicated by the currentemotional state, the controller 140 may control the feedback device 150to operate in the second mode.

At this time, the numerical value of the degree of negativity maycorrespond to a numerical value of the degree of positivity having aminus (−) value, and may correspond to a case where the emotional stateis located on a second quadrant and a third quadrant on the emotionmodel 400. In the comparison between the numerical value of the degreeof excitability and the numerical value of the degree of negativity, itis premised that the comparison is made based on the absolute value ofeach numerical value.

In addition, the controller 140 may determine the operation ratiobetween the first mode and the second mode based on the difference valuebetween the user's current emotional state and the target emotionalstate, and control the feedback device 150 based on the determinedoperation ratio for the predetermined time.

The controller 140 may determine the difference value between the user'scurrent emotional state and the target emotional state based on at leastone of the image data of the user and the bio-signal. The targetemotional state may be set when the vehicle 10 is designed, or may beset based on the user's input through the inputter 130. For example, thetarget emotional state may correspond to emotion 2, which has a lowdegree of excitability and a high degree of positivity.

As illustrated in FIG. 5 , the difference value between the user'scurrent emotional state and the target emotional state may include adifference value for the degree of excitability and a difference valuefor the degree of positivity. That is, the controller 140 may comparethe degree of excitability and the degree of positivity of the targetemotional state on the basis of the determined degree of excitabilityand the determined degree of positivity, respectively, and determine thedifference value for the degree of excitability and the difference valuefor the degree of positivity.

At this time, the controller 140 may determine the operation ratiobetween the first mode and the second mode so as to correspond to theratio between the difference value for the degree of excitability andthe difference value for the degree of positivity between the currentemotional state and the target emotional state.

Thereafter, the controller 140 may control the feedback device 150 toperform the first mode for a first time of the predetermined time basedon the determined operation ratio. The controller 140 may control thefeedback device 150 to perform the second mode for a second time of thepredetermined time, the second time equaling a difference between thepredetermined time and the first time.

That is, the ratio between the first time and the second time maycorrespond to the operation ratio determined by the controller 140.

For example, referring next to FIG. 6 , the sum of the times of a firstsection 610 and a third section 630 in which the feedback device 150operates in the first mode may be the first time. The sum of the timesof a second section 620 and a fourth section 640 in which the feedbackdevice 150 operates in the second mode may be the second time whichequals a difference between the predetermined time and the first time.

The predetermined time may be set when the vehicle 10 is designed, ormay be set based on the user's input through the inputter 130. Inaddition, the predetermined time may correspond to the estimated drivingtime to a destination calculated by the controller 140 according to theuser's destination input through the inputter 130.

The controller 140 may control the feedback device 150 such that thefirst mode and the second mode are repeatedly alternately performedbased on the predetermined number of mode switching times and thepredetermined holding time for each mode.

The predetermined number of mode switching times and the predeterminedholding time for each mode may be set when the vehicle 10 is designed,or may be set based on the user's input through the inputter 130.

For example, referring next to FIG. 6 , the predetermined number of modeswitching times may correspond to ‘3,’ and the holding time for eachmode may be set differently for each of the intervals 610, 620, 630, and640. However, the ratio of the total operation time for each mode maycorrespond to the operation ratio determined by the controller 140.However, the operation pattern of the feedback device 150 illustrated inFIG. 6 is merely an example, and various operation patterns may beprepared according to the setting.

That is, the controller 140 may control the feedback device 150 suchthat the first mode and the second mode are repeatedly alternatelyperformed. At this time, the total time that the feedback device 150operates in the first mode may correspond to the first time of thepredetermined time, and the total time that the feedback device 150operates in the second mode may correspond to a second time of thepredetermined time, the second time equaling a difference between thepredetermined time and the first time.

In addition, the controller 140 may perform an operation through aneural network on the current emotional state and the target emotionalstate, determine the number of mode switching times and the holding timefor each mode corresponding to the current emotional state and thetarget emotional state based on information characterizing the operationperformed through the neural network, and control the feedback device150 such that the first mode and the second mode are repeatedlyalternately performed based on the determined number of mode switchingtimes and the determined holding time for each mode.

Particularly, the controller 140 may perform the operation through theneural network on the determined current emotional state and the targetemotional state, determine the number of mode switching times, anddetermine an operation pattern of the feedback device 150 based on theinformation characterizing the operation performed through the neuralnetwork.

At this time, the operation pattern of the feedback device 150 maycorrespond to the mode in which the feedback device 150 operates byswitching the mode for the predetermined time based on the number oftimes an operation mode is switched for the predetermined time and theholding time for each mode.

In this case, the information characterizing the operation performedthrough the neural network may include information characterizing thenumber of mode switching times indicating the number of times the modeis switched during the predetermined time and information characterizingthe holding time for each mode indicating the time at which the mode foreach mode to be switched is held.

The above-mentioned neural network refers to the machine learning thatforms the neural structure capable of performing deep learning, so thatthe weight and bias corresponding to the configuration of the neuralnetwork continuously changes, thereby improving the reliability oflearning.

Particularly, the vehicle 10 may improve the inference result of theneural network by continuously updating the weight, bias and activationfunction included in the neural network based on the current emotionalstate, the target emotional state, and information characterizing anarrival time of the user's emotion according to the pattern of thefeedback device 150 to the target emotion. That is, the vehicle 10 maystore the determined pattern and the information characterizing thearrival time of the user's emotion to the target emotion whenever thevehicle 10 drives, and continuously update the stored neural networkbased on information characterizing the stored determined pattern andthe information characterizing the arrival time.

Meanwhile, the neural network may include the CNN that generates thefeature map output by convoluting the current emotional state and thetarget emotional state, and inputs the feature map to the neuralnetwork. However, the neural network may be performed with otherdeep-running algorithms including the RNN.

In this way, the vehicle 10 may determine the operation pattern of thefeedback device 150 that is most suitable based on the current emotionalstate and the target emotional state through the neural network that iscontinuously updated according to the use. That is, the controller 140may determine the number of mode switching times and the holding timefor each mode in which the user's emotion can quickly reach the targetemotion, based on the current emotional state and the target emotionalstate.

However, the controller 140 may control the feedback device 150 tooperate in either the first mode or the second mode based on the user'sinput through the inputter 130.

Hereinafter, the control of the feedback device 150 of the controller140 according to the modes will be described in detail.

The controller 140 may extract the emotion factor that affects theuser's current emotional state and control the feedback device 150 inthe direction of decreasing the emotion factor corresponding to thedegree of excitability of the extracted emotion factor in the firstmode, and control the feedback device 150 in the direction of increasingthe emotion factor corresponding to the degree of positivity of theextracted emotion factors in the second mode.

Referring next to FIG. 7 , the feedback device 150 may include at leastone of the speaker 151, the display 152, the air conditioner 154provided in the vehicle 10, and the vibration device 153 provided in theseat of the vehicle 10.

The speaker 151 may output sound to the inside of the vehicle 10 havinga frequency corresponding to the target breathing cycle or the targetheartbeat cycle in the first mode under the control of the controller140, or output sound to the inside of the vehicle 10 that increases theemotion factor corresponding to the degree of positivity in the secondmode under the control of the controller 140.

That is, the controller 140 may set the breathing cycle and theheartbeat cycle when the target emotional state is the target breathingcycle and the target heartbeat cycle, respectively, and control thespeaker 151 to output sound corresponding to the target breathing cycleor the target heartbeat cycle to the inside of the vehicle 10 so thatthe degree of excitability of the user is decreased in the first mode.Accordingly, the vehicle 10 can cause the user's breathing cycle to beinduced to the target breathing cycle or the user's heartbeat cycle tobe induced to the target heartbeat cycle.

At this time, the sound that increases the emotion factor correspondingto the degree of positivity may be set when the vehicle 10 is designed,and may correspond to sound that has at least one of the size, thegenre, the equalizer, the tone color, and the sound wave region of thesound in which the user can feel the positive emotion. For example, thesound that increases the emotion factor corresponding to the degree ofpositivity may include hip-hop music, classical music, and pop music inwhich the user can feel the positive emotion. However, the sound thatincreases the emotion factor corresponding to the degree of positivitymay be set based on the user's input through the inputter 130. That is,the user may set the sound that caused the positive emotion to be thesound that increases the emotion factor corresponding to the degree ofpositivity, and the sound that is set can be stored in the storage 160.

The display 152 according to embodiments of the present disclosure mayoutput the image having a frequency corresponding to the targetbreathing cycle or the target heartbeat cycle to the inside of thevehicle 10 in the first mode, and output the image having the frequencycorresponding to the sound for increasing the emotion factorcorresponding to the degree of positivity to the inside of the vehicle10 in the second mode, under the control of the controller 140.

More particularly, the controller 140 may control the display 152 toincrease or decrease the intensity of light output according to thefrequency corresponding to the target breathing cycle or the targetheartbeat cycle in the first mode, and control the display 152 toincrease or decrease the intensity of the light output according to thefrequency corresponding to the sound that increases the emotion factorcorresponding to the degree of positivity in the second mode.

The display 152 may also output the image of a color that decreases theemotion factor corresponding to the degree of excitability in the firstmode based on the control of the controller 140, and output the image ofthe color that increases the emotion factor corresponding to thepositivity of the image in the second mode, under the control of thecontroller 140.

At this time, the color for decreasing the emotion factor correspondingto the degree of excitability and the color for increasing the emotionfactor corresponding to the degree of positivity may be set when thevehicle 10 is designed, may be set by the controller 140 based oncorrelation information between the color and the emotion factorobtained from the external server, and may be set by the user throughthe inputter 130.

The vibration device 153 according to embodiments of the presentdisclosure may output the vibration having a frequency corresponding tothe target breathing cycle or the target heartbeat cycle in the firstmode, or output the vibration having the frequency corresponding to thesound for increasing the emotion factor corresponding to the degree ofpositivity in the second mode, under the control of the controller 140.

At this time, the outputted vibration causes the seat provided in thevehicle 10 to vibrate, and may be transmitted to the user located on theseat through the seat.

The air conditioner 154 may output at least one of the perfume and windthat decreases the emotion factor corresponding to the degree ofexcitability in the first mode, and output at least one of the perfumeand wind that increases the emotion factor corresponding to the degreeof positivity in the second mode, under the control of the controller140.

At this time, the perfume and wind that decrease the emotion factorcorresponding to the degree of excitability may be set when the vehicle10 is designed, may be set by the controller 140 based on thecorrelation information between the perfume and the emotion factorobtained from the external server, and may be set by the user throughthe inputter 130. For example, the perfume and wind that decrease theemotion factor corresponding to the degree of excitability maycorrespond to jasmine perfume and a breeze, respectively. However, thepresent disclosure is not limited thereto, and any perfume that candecrease the emotion factor corresponding to the degree of excitabilitymay be included without limitation. Also, the wind that decreases theemotion factor corresponding to the degree of excitability maycorrespond to wind having at least one of wind direction, intensity, andtemperature that can lower the degree of excitability.

In addition, the perfume and wind that decrease the emotion factorcorresponding to the degree of positivity may be set when the vehicle 10is designed, may be set by the controller 140 based on the correlationinformation between the perfume and the emotion factor obtained from theexternal server, and may be set by the user through the inputter 130.For example, the perfume and wind that increase the emotion factorcorresponding to the degree of positivity may correspond to lemonperfume and a cool breeze, respectively. However, the present disclosureis not limited thereto, and any perfume that can increase the emotionfactor corresponding to the degree of positivity may be included withoutlimitation. Also, the wind that increases the emotion factorcorresponding to the degree of positivity may correspond to wind havingat least one of wind direction, intensity, and temperature that canraise the degree of positivity.

In this way, the controller 140 may control at least one of the speaker151, the display 152, the vibration device 153, and the air conditioner154 corresponding to the feedback device 150 to induce the user'spositive emotion and decrease the degree of excitability of the user.

Hereinafter, a control method of the vehicle 10 according to embodimentsof the present disclosure will be described. The vehicle 10 according tothe above-described embodiment of the present disclosure may be appliedto the control method of the vehicle 10, as will be described below.Therefore, descriptions given above with reference to FIGS. 1 to 7 maybe applied to the control method of the vehicle 10 in the same manner,unless otherwise noted.

FIG. 8 is a flowchart illustrating a method of controlling the feedbackdevice in a method of controlling the vehicle according to embodimentsof the present disclosure.

As shown in FIG. 8 , the vehicle 10 according to embodiments of thepresent disclosure may determine information characterizing the user'scurrent emotional state based on at least one of the bio-signal or theimage information (810).

Particularly, the controller 140 may determine the facial expression ofthe user based on the image data of the user, and obtain informationcharacterizing the emotional state corresponding to the facialexpression. At this time, the image information may be obtained by thecamera 110 provided in the vehicle 10.

In addition, the controller 140 may obtain information characterizingthe user's emotional state corresponding to the bio-signal of the userbased on the bio-signal of the passenger 200. At this time, thebio-signal may be obtained by the bio-signal sensor 120 provided in thevehicle 10.

At this time, the information characterizing the current emotional statemay be indicated by the numerical value in which each degree ofpositivity and degree of excitability is expressed numerically accordingto the degrees.

The vehicle 10 may compare the degree of excitability of the user andthe degree of positivity of the user indicated by the current emotionalstate (820).

Particularly, the controller 140 may compare the degree of excitabilityof the user and the degree of positivity of the user indicated by thecurrent emotional state, and select either the first mode forcontrolling the feedback device 150 in a direction of decreasing thedegree of excitability of the user or the second mode for controllingthe feedback device 150 in a direction of increasing the degree ofpositivity of the user based on the comparison result. That is, thecontroller 140 may determine the operation ratio between the first modeand the second mode to 100 to 0 or 0 to 100 for immediate feedback onthe current emotional state according to the embodiment. Accordingly,the controller 140 may control the feedback device 150 to operate ineither the first mode or the second mode.

Particularly, the controller 140 may compare the numerical value of thedegree of excitability of the user with the numerical value of thedegree of negativity of the user, i.e., the minus (−) degree ofpositivity, based on information characterizing the current emotionalstate.

At this time, the numerical value of the degree of negativity maycorrespond to a numerical value of the degree of positivity having aminus (−) value, and may correspond to a case where the emotional stateis located on a second quadrant and a third quadrant on the emotionmodel 400. In the comparison between the numerical value of the degreeof excitability and the numerical value of the degree of negativity, itis premised that the comparison is made based on the absolute value ofeach numerical value.

The vehicle 10 may select either the first mode for controlling thefeedback device 150 in a direction of decreasing the degree ofexcitability of the user or the second mode for controlling the feedbackdevice 150 in a direction of increasing the degree of positivity of theuser based on the comparison result (830).

In this case, when the numerical value of the degree of excitabilityindicated by the current emotional state is higher than the numericalvalue of the degree of negativity indicated by the current emotionalstate, the controller 140 may control the feedback device 150 to operatein the first mode. When the numerical value of the degree ofexcitability indicated by the current emotional state is lower than thenumerical value of the degree of negativity indicated by the currentemotional state, the controller 140 may control the feedback device 150to operate in the second mode.

That is, the controller 140 may determine the operation ratio betweenthe first mode and the second mode to 100 to 0 or 0 to 100 for immediatefeedback on the current emotional state according to the embodiment.Accordingly, the controller 140 may control the feedback device 150 tooperate in either the first mode or the second mode.

In addition, the vehicle 10 may control the feedback device 150 based onthe selected mode (840).

That is, the controller 140 may control at least one of the speaker 151,the display 152, the vibration device 153, and the air conditioner 154corresponding to the feedback device 150 to induce the user's positiveemotion and decrease the degree of excitability of the user.

The configuration in which the controller 140 controls the feedbackdevice 150 is the same as that described above, thus a detailedexplanation will be omitted.

Hereinafter, a control method of the vehicle 10 for the case where thevehicle 10 may determine the operation ratio between the first mode andthe second mode based on the difference value between the user's currentemotional state and the target emotional state, and control the feedbackdevice 150 based on the determined operation ratio for the predeterminedtime will be described.

FIG. 9 is another flowchart illustrating a method of controlling thefeedback device in a method of controlling the vehicle according toembodiments of the present disclosure.

As shown in FIG. 9 , the vehicle 10 according to embodiments of thepresent disclosure may determine information characterizing the user'scurrent emotional state based on at least one of the bio-signal or theimage information (910).

Particularly, the controller 140 may determine the facial expression ofthe user based on the image data of the user, and obtain informationcharacterizing the emotional state corresponding to the facialexpression. At this time, the image information may be obtained by thecamera 110 provided in the vehicle 10.

In addition, the controller 140 may obtain information characterizingthe user's emotional state corresponding to the bio-signal of the userbased on the bio-signal of the passenger 200. At this time, thebio-signal may be obtained by the bio-signal sensor 120 provided in thevehicle 10.

At this time, the information characterizing the current emotional statemay be indicated by the numerical value in which each of the degree ofpositivity and degree of excitability is expressed numerically accordingto the degrees.

The vehicle 10 may determine the difference value for the degree ofexcitability and the difference value for the degree of positivitybetween the current emotional state and the target emotional state(920).

That is, the vehicle 10 may determine the difference value between theuser's current emotional state and the target emotional state based onat least one of the image data of the user and the bio-signal. Thetarget emotional state may be set when the vehicle 10 is designed, ormay be set based on the user's input through the inputter 130. Forexample, the target emotional state may correspond to emotion 2, whichhas the low degree of excitability and the high degree of positivity.

The difference value between the user's current emotional state and thetarget emotional state may include the difference value for the degreeof excitability and the difference value for the degree of positivity.That is, the controller 140 may compare the degree of excitability andthe degree of positivity of the target emotional state on the basis ofthe determined degree of excitability and the determined degree ofpositivity, respectively, and determine the difference value for thedegree of excitability and the difference value for the degree ofpositivity.

The vehicle 10 may determine the operation ratio between the first modeand the second mode so as to correspond to the ratio between thedifference value for the degree of excitability and the difference valuefor the degree of positivity (930).

In addition, the vehicle 10 may control the feedback device 150 for thepredetermined time based on the determined operation ratio (940).

Particularly, the controller 140 may control the feedback device 150 toperform the first mode for the first time of the predetermined timebased on the determined operation ratio. The controller 140 may controlthe feedback device 150 to perform the second mode for the second timewhich equals a difference between the predetermined time and the firsttime.

That is, the ratio between the first time and the time excluding thefirst time of the predetermined time may correspond to the operationratio determined by the controller 140.

The predetermined time may be set when the vehicle 10 is designed, ormay be set based on the user's input through the inputter 130. Inaddition, the predetermined time may correspond to the estimated drivingtime to a destination calculated by the controller 140 according to theuser's destination input through the inputter 130.

The controller 140 may control the feedback device 150 such that thefirst mode and the second mode are repeatedly alternately performedbased on the predetermined number of mode switching times and thepredetermined holding time for each mode.

The predetermined number of mode switching times and the predeterminedholding time for each mode may be set when the vehicle 10 is designed,or may be set based on the user's input through the inputter 130.

That is, the controller 140 may control the feedback device 150 suchthat the first mode and the second mode are repeatedly alternatelyperformed. At this time, the total time that the feedback device 150operates in the first mode may correspond to the first time of thepredetermined time, and the total time that the feedback device 150operates in the second mode may correspond to a second time of thepredetermined time, the second time equaling a difference between thepredetermined time and the first time.

In addition, the controller 140 may perform an operation through aneural network on the current emotional state and the target emotionalstate, determine the number of mode switching times and the holding timefor each mode corresponding to the current emotional state and thetarget emotional state based on information characterizing the operationperformed through the neural network, and control the feedback device150 such that the first mode and the second mode are repeatedlyalternately performed based on the determined number of mode switchingtimes and the determined holding time for each mode.

Particularly, the controller 140 may perform the operation through theneural network on the determined current emotional state and the targetemotional state, determine the number of mode switching times, anddetermine the operation pattern of the feedback device 150 based on theinformation characterizing the operation performed through the neuralnetwork.

At this time, the operation pattern of the feedback device 150 maycorrespond to the mode in which the feedback device 150 operates byswitching the mode for the predetermined time based on the number oftimes an operation mode is switched for the predetermined time and theholding time for each mode.

In this case, the information characterizing the operation performedthrough the neural network may include information characterizing thenumber of mode switching times indicating the number of times the modeis switched during the predetermined time and information characterizingthe holding time for each mode indicating the time at which the mode foreach mode to be switched is held.

The above-mentioned neural network refers to the machine learning thatforms the neural structure capable of performing deep learning, so thatthe weight and bias corresponding to the configuration of the neuralnetwork continuously changes, thereby improving the reliability oflearning.

Particularly, the vehicle 10 may improve the inference result of theneural network by continuously updating the weight, bias and activationfunction included in the neural network based on the current emotionalstate, the target emotional state, and information characterizing anarrival time of the user's emotion according to the pattern of thefeedback device 150 to the target emotion. That is, the vehicle 10 maystore the determined pattern and the information characterizing thearrival time of the user's emotion to the target emotion whenever thevehicle 10 drives, and continuously update the stored neural networkbased on information characterizing the stored determined pattern andthe information characterizing the arrival time.

In this way, the vehicle 10 may determine the operation pattern of thefeedback device 150 that is most suitable based on the current emotionalstate and the target emotional state through the neural network that iscontinuously updated according to the use. That is, the controller 140may determine the number of mode switching times and the holding timefor each mode in which the user's emotion can quickly reach the targetemotion, based on the current emotional state and the target emotionalstate.

The configuration in which the controller 140 controls the feedbackdevice 150 is the same as that described above, thus a detailedexplanation will be omitted.

According to the vehicle and the method for controlling the vehicle asdescribed above, by determining a mode in which the user's emotionalstate in the vehicle is determined as a degree of positivity and adegree of excitability, and determining an operation ratio between amode for controlling a feedback device so as to decrease the degree ofexcitability and a mode for controlling the feedback device so as toincrease the degree of positivity according to the determined degree ofpositivity and degree of excitability, the user's emotion can bepromptly caused to the target emotion.

The exemplary embodiments of the present disclosure have thus far beendescribed with reference to the accompanying drawings. It will beobvious to those of ordinary skill in the art that the presentdisclosure may be practiced in other forms than the exemplaryembodiments as described above without changing the technical idea oressential features of the present disclosure. The above exemplaryembodiments are only by way of example, and should not be interpreted ina limited sense.

What is claimed is:
 1. A vehicle comprising: a feedback device; abio-signal sensor configured to measure a bio-signal of a user; and acontroller operatively coupled to the feedback device and the bio-signalsensor, the controller including a memory configured to store at leastone program instruction and processor configured to execute the at leastone program instruction, and the controller being configured to:determine information characterizing a current emotional state of theuser based on the bio-signal; calculate, based on a difference valuebetween the current emotional state and a target emotional state, anoperation ratio between a first mode for controlling operation of thefeedback device to decrease a degree of excitability of the user and asecond mode for controlling the operation of the feedback device toincrease a degree of positivity of the user; control the operation ofthe feedback device for a predetermined time based on the operationratio; calculate a first difference value between the degree ofexcitability for the current emotional state and the degree ofexcitability for the target emotional state calculate; calculate asecond difference value between the degree of positivity for the currentemotional state and the degree of positivity for the target emotionalstate calculate; and calculate the operation ration between the firstmode and the second mode as equivalent to a ratio between the firstdifference value and the second difference value.
 2. The vehicleaccording to claim 1, wherein the controller is configured to controlthe operation of the feedback device, based on the operation ratio, suchthat the feedback device performs the first mode for a first time of thepredetermined time and performs the second mode for a second time of thepredetermined time, wherein the second time equals a difference betweenthe predetermined time and the first time.
 3. The vehicle according toclaim 2, wherein the controller is configured to control the operationof the feedback device such that the first mode and the second mode arerepeatedly alternately performed based on a predetermined number of modeswitching times and a predetermined holding time for each mode.
 4. Thevehicle according to claim 1, wherein the controller is configured todetermine the degree of excitability of the user and the degree ofpositivity of the user based on the current emotional state, to comparethe degree of excitability of the user with the degree of positivity ofthe user, and to perform either the first mode or the second mode basedon the comparison of the degree of excitability of the user with thedegree of positivity of the user.
 5. The vehicle according to claim 1,wherein the feedback device is disposed in the vehicle and comprises atleast one of a speaker, a display, an air conditioner, and a vibrationdevice installed in a seat.
 6. The vehicle according to claim 1, whereinthe controller is configured to: extract an emotion factor that affectsthe current emotional state; in the first mode, control the operation ofthe feedback device causing an emotion factor to decrease according tothe degree of excitability of the extracted emotion factor; and in thesecond mode, control the operation of the feedback device causing anemotion factor to increase according to the degree of positivity of theextracted emotion factor.
 7. A vehicle comprising: a feedback device; abio-signal sensor configured to measure a bio-signal of a user; and acontroller operatively coupled to the feedback device and the bio-signalsensor, the controller including a memory configured to store at leastone program instruction and processor configured to execute the at leastone program instruction, and the controller being configured to:determine information characterizing a current emotional state of theuser based on the bio-signal; calculate, based on a difference valuebetween the current emotional state and a target emotional state, anoperation ratio between a first mode for controlling operation of thefeedback device to decrease a degree of excitability of the user and asecond mode for controlling the operation of the feedback device toincrease a degree of positivity of the user; control the operation ofthe feedback device for a predetermined time based on the operationratio; control the operation of the feedback device, based on theoperation ratio, such that the feedback device performs the first modefor a first time of the predetermined time and performs the second modefor a second time of the predetermined time, wherein the second timeequals a difference between the predetermined time and the first time;perform an operation through a neural network based on the currentemotional state and the target emotional state; determine the number ofmode switching times and the holding time for each mode corresponding tothe current emotional state and the target emotional state based oninformation characterizing the operation performed through the neuralnetwork; and control the operation of the feedback device such that thefirst mode and the second mode are repeatedly alternately performedbased on the determined number of mode switching times and thedetermined holding time for each mode.
 8. A method for controlling avehicle, the vehicle including a feedback device, a bio-signal sensorconfigured to measure a bio-signal of a user, and a controlleroperatively coupled to the feedback device and the bio-signal sensor,the method comprising: determining, by the controller, informationcharacterizing a current emotional state of the user based on thebio-signal; calculating, by the controller, based on a difference valuebetween the current emotional state and a target emotional state, anoperation ratio between a first mode for controlling the operation ofthe feedback device to decrease a degree of excitability of the user anda second mode for controlling the operation of the feedback device toincrease a degree of positivity of the user; and controlling, by thecontroller, the operation of the feedback device for a predeterminedtime based on the operation ratio; wherein the calculating of theoperation ratio comprises: calculating a first difference value betweenthe degree of excitability for the current emotional state and thedegree of excitability for the target emotional state calculate;calculating a second difference value between the degree of positivityfor the current emotional state and the degree of positivity for thetarget emotional state calculate; and calculating the operation ratiobetween the first mode and the second mode as equivalent to a ratiobetween the first difference value and the second difference value. 9.The method according to claim 8, wherein the feedback device is disposedin the vehicle and comprises at least one of a speaker, a display, anair conditioner, and a vibration device installed in a seat.
 10. Themethod according to claim 8, wherein the controlling of the operation ofthe feedback device comprises: extracting an emotion factor that affectsthe current emotional state; in the first mode, controlling theoperation of the feedback device causing an emotion factor to decreaseaccording to the degree of excitability of the extracted emotion factor;and in the second mode, controlling the operation the feedback devicecausing an emotion factor to increase according to the degree ofpositivity of the extracted emotion factor.
 11. A method for controllinga vehicle, the vehicle including a feedback device, a bio-signal sensorconfigured to measure a bio-signal of a user, and a controlleroperatively coupled to the feedback device and the bio-signal sensor,the method comprising: determining, by the controller, informationcharacterizing a current emotional state of the user based on thebio-signal; calculating, by the controller, based on a difference valuebetween the current emotional state and a target emotional state, anoperation ratio between a first mode for controlling the operation ofthe feedback device to decrease a degree of excitability of the user anda second mode for controlling the operation of the feedback device toincrease a degree of positivity of the user; and controlling, by thecontroller, the operation of the feedback device for a predeterminedtime based on the operation ratio; wherein the controlling of thefeedback device comprises controlling the operation of the feedbackdevice, based on the operation ratio, such that the feedback deviceperforms the first mode for a first time of the predetermined time andperforms the second mode for a second time of the predetermined time,wherein the second time equals a difference between the predeterminedtime and the first time.
 12. The method according to claim 11, whereinthe controlling of the operation of the feedback device comprises:controlling the operation of the feedback device such that the firstmode and the second mode are repeatedly alternately performed based on apredetermined number of mode switching times and a predetermined holdingtime for each mode.
 13. The method according to claim 11, wherein thecontrolling of the operation of the feedback device comprises:performing an operation through a neural network based on the currentemotional state and the target emotional state; determining the numberof mode switching times and the holding time for each mode correspondingto the current emotional state and the target emotional state based oninformation characterizing the operation performed through the neuralnetwork; and controlling the operation of the feedback device such thatthe first mode and the second mode are repeatedly alternately performedbased on the determined number of mode switching times and thedetermined holding time for each mode.
 14. A method for controlling avehicle, the vehicle including a feedback device, a bio-signal sensorconfigured to measure a bio-signal of a user, and a controlleroperatively coupled to the feedback device and the bio-signal sensor,the method comprising: determining, by the controller, informationcharacterizing a current emotional state of the user based on thebio-signal; calculating, by the controller, based on a difference valuebetween the current emotional state and a target emotional state, anoperation ratio between a first mode for controlling the operation ofthe feedback device to decrease a degree of excitability of the user anda second mode for controlling the operation of the feedback device toincrease a degree of positivity of the user; and controlling, by thecontroller, the operation of the feedback device for a predeterminedtime based on the operation ratio; wherein the controlling of theoperation of the feedback device comprises: determining the degree ofexcitability of the user and the degree of positivity of the user basedon the current emotional state; comparing the degree of excitability ofthe user with the degree of positivity of the user; and performingeither the first mode or the second mode based on the comparison of thedegree of excitability of the user with the degree of positivity of theuser.